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ABSTRACT 

The development of a scoring tool to be used with 
alternative and authentic assessment strategies is discussed, 
describing the parts of such a tool and the development process. The 
design is based on the Innovation Configuration (IC) process of the 
Concerns“Bas ed Adoption Model. To begin, components of the test task 
are identified. Variations that accomplish the various components are 
then determined. A quality line is established that sets least 
desirable variations on the left side of the tool and most desirable 
variations on the right side. The quality line may be percentages of 
student accomplishment, or it may be developed from judgment of 
student work or grade level performance. An IC-rubric (IC“R) is 
developed from combinations of components and variations for any 
performance task. Components are listed on the y~axis and variations 
from least to most desirable (quality line) on the x“axis. An 
activity for developing and applying an IC"R is included. (Contains 
six inserts and four references.) (SLD) 
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DESIGNING SCORING TOOLS FOR AUTHENTIC AND 
ALTERNATIVE ASSESSMENTS: A COMMON SENSE METHOD 



Raymond Francis, Ed.D. 

Appalachia Educational Laboratory 
Charleston, WV 

Assessment! Let's face it, the term assessment is loaded. It is loaded with the power to 
determine students' level of progress, the degree of student proficiency, and the state of competency 
reached as a result of participation in classroom instmction. In our schools today, the application 
of the term assessment still most commonly refers to tests, homework, and that most infamous of 
assignments, the vocabulary quiz. It is time for a change! 

Now is the time for educators to address student assessment and make it meaningful to the 
student and the curriculum, make it reflective of high expectations, and make it related to real world 
experiences relevant to students' lives (Pandey, 1990). As what has been called the "driving force 
promoting change in schools" (Field, 1991), classroom teachers must make the conscious choice to 
develop student assessment in conjunction with changes in curriculum and instructional strategies. 
To that end, our approach is simple in both design and application. 

Designing Scoring Tools 

The design is based on the Innovation Configuration (1C) process created as a part of the 
Concerns-Based Adoption Model (CBAM) (Hall & Hord, 1987). The concept of Innovation 
Configuration in the model piovides the theory and foundation for creating assessment plans based 
on teacher decisions about what is important in their individual classroom. The application is based 
on a common sense approach determined by the primary force in the classroom: the teacher. 
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There are a variety of approaches to designing assessment. They include portfolios, tasks, 
problem design and a variety of others. To be effective, each form, or strategy, of assessment must 
be driven by a plan and involves several parts. Those parts include: setting a goal or objective for 
student performance, development of a task prompt, assignment of a student task, development of 
an assessment tool, and scoring of student performance on the task. This paper deals wholly v'ith 
the latter part, the development of a scoring tool to be used with alternative and authentic assessment 
strategies. The parts of such a scoring tool are described, and the process for developing the tool is 
outlined in the sections that follow. 

Defining the Parts of the Scoring Tool 

Scoring tools appear to function on many levels with several common parts (Bracey, G. & 
Pool, C., 1992). The first of three components is the "what." That is, "what" are we looking for 
students to be able to do. This might take the form of the student sorting a series of objects by a 
named attribute, designing a map of the neighborhood or playground, solving a problem, or creating 
an alternative to an existing answer. The second and third components are the "how" and the 
"quality line." 

Components of the Task 

The "whats" that are referred to in this context are spvccifically named the "COMPONENT" 
for the working purposes of this paper. The term "component" is borrowed from Taking Charge of 
Chang e (Hord, Hall, Rutherford, and Huling- Austin, 1987). The authors' long term work with the 
use of the IC, with practitioners, lends practical theory to the development of a process, a strategy 
to develop meaningful student assessments. To use a common example for demonstrating the term 
component, a sample task might be to create a scale 

model of the solar system which includes all planets, the sun, and at least two unique identifying 
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features for each planet. Components of this task include, but are not limited to; 

• Creation of the model with the sun and all planets represented 

• The model is represented at scale 

• The model has at least two identifying features for each planet 



INSERT 1 



Variations of a Component 

The second part of the scoring tool is the "how." How is the component accomplished? How 
do versions of accomplishment of the component compare? How do the versions compare to 
accepted standards or benchmarks? The how versions, or "VARIATIONS" as they are referred to, 
are intended to be objective and observable on a continuum of possibilities. Each component may 
be demonstrated by students in the classroom with different variations that have very different 
degrees of proficiency or performance. In the above example. Component #1 may have several 
variations which include: 

INSERT 2 

Component #2 may have the following variations: 

INSERT 3 



Component #3 may have the following variations: 

INSERT 4 



It is important to note at this point that you, the reader, may consider other variations and 
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components as being essential for the assessment of this task. That is fine. As educators, we all 
bring unique perspectives to the development of assessment tools for the classroom. There is no one 
right number of components or variations for a task. The component variations should represent the 
criteria for the task's appropriate accomplislmient. 

The Quality Line 

The third component of the scoring tool is the "QUALITY LINE." A constant pattern is set 
on the matric placing least desirable variations on the left side of the tool and ranging to the most 
desirable, or highest quality, variations on the right side of the tool. 

The quality line may be developed for a number of reasons and for a variety of purposes. 
The quality line may represent percentages of student accomplishment. For example: 

60 % 70 % 80 % 90 % 100 % 

Or, the quality line may be developed from judgment of student work. For example; 

poor average above average 

Or. the quality line may indicate grade level performance. For example; 

below grade level at grade level above grade level 
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Regardless of how the quality line is constructed, it should relate to the components and variations 
developed for the performance task and help the teacher assess the student. 

We have created an Innovation Configuration-Rubric (IC-R) that was constructed from the 
combinations of components and variations developed for any given performance task. The 
Innovation Configuration-Rubric (IC-R) takes the form of a standard matrix or rubric with the 

components listed along the y-axis and the variations from least to most desirable listed on the x- 
axis. 

The IC-R for our example about the model of the solar system might look in its entirety like 
the following: 



INSERT 5 



Developing Your Own IC-R 

As you have seen, the development of an IC-R is relatively easy but requires thoughtfulness 
and careful consideration. To create an IC-R for yourself, try the following activity. 

1. First, look at your curriculum. What is something you expect your students to be 
able to do? Write it here: ^ 
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2. Identify a task that is a part of your answer in #1 . 

3 . Use the box below and list three components of the task from #2. We are using thiee 
components for the purpose of this paper; there could, of course, be more than three. 

4. For each component identify three possible variations of ways in which the 
component may be performed. 
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5. Next, fill in the quality line to help you judge the quality of the variations and better 
assess performance on the task. You should use a scale or indicator you are 
comfortable with, such as unsatisfactory/satisfactory/outstanding or poor/ average/ 
above average. 

INSERT 6 

6. After completing the IC-R you should check your IC-R by asking yourself some 
simple questions. 

a. Is this component something I want to assess about my students' 
performance? 

b. Are the variations likely to be in my classroom and does one variation 
represent highest quality? 

c. Are the components/variations observable? 

After asking yourself these questions eliminate any components or variations to which you 
have answered "no" as a response. 



Application of the IC-R 

One big question educators have about the IC-R is "How do I use the IC-R to assess student's 
learning?" The answer we give is "through a holistic process." The use of a holistic process 
provides the most accurate strategy while maintaining a view of the entire task. 

Let's go back to the example about the model of the solar system. Think about the IC-R 
assessment tool and this example of a student's performance on the task: created a model without 
a sun, was lacking the planets of Venus and Pluto, only three of the planets had two identifiable 




6 



8 



features, and Earth and Mars were larger than Saturn and Neptune. Go to the sample IC-R and shade 
in the variations listed. What is the students' performance? If you compare the shaded-in variations 
to the quality line, it becomes easy to see that the students' performance on the task was below the 
satisfactory level, or at the unsatisfactory level. 

However, if the student were to have included the sun and all of the planets, made the 
representation of each planet relative to the others, and included two or more identifying features on 
each planet, the student would be in the high end or outstanding portion of the quality line. 

As we move into the performance task and portfolio assessment era in education, we must 
keep in mind that the purpose of assessment is to help improve student performance inform and 
improve teachers' practice. It is up to the individual teacher to create and use the IC-R in a manner 
which is meaningful to the student and functional in the classroom. 
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COMPONENTS 




1 . Create a model of the 
solar system containing the 
sun and all the planets. 




2. Make all parts of the 
model at a relative scale. 




3. Include at least two 
identifying features for each 
planet. 
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INSERT 2 



COMPONENT 

#1 


Variation Variation Variauoo Variation 

A B C li 


Create a model 
v/ith the sun and 
all planets 
represented. 


The model has 
not been 
created. 


The model 
lacks the sun. 


The model 
lacks one 
or more of 
the planets. 


The sun and 
all planets 
are repre- 
sented. 
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COMPONENT 

#2 


Variation A Variation B Variation C 


The model is 
represented at 
scale. 


The model is not 
represented at 
scale. 


The model is 
represented at scale 
in terms of either 
planet size or 
distance between 
them. 


The model is 
represented at 
scale in terms of 
the size of the 
planets and the 
distance between 
them. 
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COMPONENT 

#3 


Variation A Variation B Variation C Variation D 


The model has 
at least two 
identifying | 

features for 1 

each planet. | 


No planets have 
identifying 
1 features. 

1 


Some planets 
have two 
identifying 
features. 


All planets 
have one 
identifying 
feature. 


All planets 
have 

at least two 
identifying 
features. 
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Unsatisfactory 



Satisfactory Outstanding 



Assessment Line 



CO^^PONENTS j 


1 VARIATIONS 


1 . Create a 
model of the 
solar system 
containing the 
sun and all 
the planets. 


a. No model b. The model c. The model d. The sun and 

was created. does not include lacks one all the planets 

the sun. or more of are represented, 

the planets. 


2. Make all 
parts of the 
model at a 
relative scale. 


a. The model is not represented in b. The model is represented in a 

a relative scale. relative scale. 


3. Include at 
least two 
identifying 
features for 
each planet. 


a. None of the b. Some of c. Most of the d. All of the 

planets have the planets have planets have planets have 

two identifying two identifying two two identi- 

features. features. identifying lying features. 

features. 
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(Quality Line) 



COMPONENTS | 


1 VARIATIONS 


Component 1; 


a. 


b. 


c. 


Component 2: 


a. 


b. 


c. 


Component 3 


a. 


b. 


c. 
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